Skip to content

test: Avoid hang in big segment status polling specs - #440

Merged
jsonbailey merged 1 commit into
mainfrom
devin/1789424560-jruby-bigsegments-hang
Sep 16, 2026
Merged

jsonbailey merged 1 commit into
mainfrom
devin/1789424560-jruby-bigsegments-hang

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Sep 14, 2026

Copy link
Copy Markdown
Member

Fixes the intermittent JRuby CI hang by removing a race in the big segment status polling specs, and adds a job timeout so a future hang fails fast instead of running until GitHub's limit.

  • spec/impl/big_segments_spec.rb: the two status polling examples no longer assume the first status change is observed after add_observer
  • .github/workflows/build-gem.yml: timeout-minutes: 20 on the gem build job

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions
Implementation details

Root cause

BigSegmentStoreManager.new starts its status poll worker immediately (start_delay of 0), and BigSegmentStoreStatusProviderImpl#update_status only notifies observers when the status differs from the previous one. Both status polling examples constructed the manager, then added an observer, then blocked on Queue#pop for the initial status. If the worker's first poll ran before add_observer, the initial status was already recorded, no notification was ever sent, and the unbounded Queue#pop blocked forever — taking the whole rspec process with it, so RuboCop and the gem build never ran and the job sat until it was cancelled at the 6 hour limit.

MRI generally loses that race (a newly spawned thread does not run until the main thread yields), which is why this only showed up on the JRuby job, where threads run truly concurrently.

Solution

The examples now read the initial status directly from status_provider.status and use a helper that waits, with a timeout, for the next status matching a predicate. That makes them independent of whether a status was queued before the observer was registered, and a broken expectation now fails instead of hanging.

The job timeout is defense in depth: any other future hang in tests, rubocop, or contract tests now fails the job in minutes.

Alternatives considered

Changing the manager to delay the first poll or to always notify on the first status would make the specs pass, but that changes SDK behavior to accommodate a test, and the current behavior (no notification unless the status changes) is intended.

Testing

Reproduced the hang locally on JRuby 9.4.15.0 (2 pinned cores), where the status polling group hung every attempt before the change and passed repeatedly after it. The full suite (bundle exec rspec spec --tag '~flaky') passes on JRuby with the persistent stores running.

Known remaining issue (not addressed here)

ld-eventsource connection threads can still outlive an example and call into an expired rspec double via StreamProcessor#log_connection_result, which prints an ExpiredTestDoubleError from a terminating thread. It is noisy but does not hang or fail the suite; worth a separate look.

Link to Devin session: https://app.devin.ai/sessions/700251d712274fcfaabc41ae8e479b3f
Open in Devin Desktop: https://app.devin.ai/desktop/session/700251d712274fcfaabc41ae8e479b3f?variant=devin
Requested by: @kinyoklion


Note

Overview
Fixes intermittent JRuby CI hangs in big segment status polling tests and caps gem build job runtime so future stalls fail fast.

The two status polling examples no longer block on the first Queue#pop after registering an observer—a race on JRuby where the poll worker can finish before the observer is attached, so no notification fires and the test waits forever. They now assert the initial state via status_provider.status and use a new next_status_matching helper (5s timeout, predicate filter) for subsequent transitions.

The build-gem workflow job gets timeout-minutes: 20 as defense in depth.

Reviewed by Cursor Bugbot for commit c9aac8f. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: rlamb@launchdarkly.com <4955475+kinyoklion@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration

Copy link
Copy Markdown
Contributor

@cursor review

@kinyoklion
kinyoklion marked this pull request as ready for review September 15, 2026 15:44
@kinyoklion
kinyoklion requested a review from a team as a code owner September 15, 2026 15:44
@jsonbailey
jsonbailey merged commit b0627ec into main Sep 16, 2026
12 checks passed
@jsonbailey
jsonbailey deleted the devin/1789424560-jruby-bigsegments-hang branch September 16, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants